Reversing and Smoothing the Multinomial Naive Bayes Text Classifier

Abstract. The naive Bayes text classifier has long been a core technique in information retrieval and, more recently, it has emerged as a focus of research itself in machine learning. This paper is concerned with the naive Bayes text classifier in its multinomial model instantiation. This model and an “equivalent” reversed version proposed here are interpreted under the statistical framework of log-linear modelling. The reversed version provides an alternative way for parameter estimation, which (in a broad sense) is actually the main issue considered. The paper is to a large extent devoted to the study of the effects of parameter smoothing and document length normalization. For the purpose of parameter smoothing, we consider a standard smoothing method for text classification and two alternative techniques that are often used in the context of statistical language modelling for speech recognition. Empirical results are provided comparing these techniques and the effect of length normalization for both the multinomial model and its reversed version.